High-frequency simulations of seismic wave propagation in the whole Earth on 150,000 processor cores of a petaflop machine

SPECFEM3D GLOBE is a spectral-element application enabling the simulation of global seismic wave propagation in the 3D Earth at unprecedented resolution. A fundamental challenge in global seismology is to model the propagation of waves with periods between 1 and 2 seconds, the highest frequency signals that can propagate clear across the Earth. These waves help reveal the 3D structure of the Earth’s deep interior and can be compared to seismographic recordings. We broke the 2 second barrier using the 62K processor Ranger system at TACC. We obtained similar results on the XT4 Franklin system at NERSC and the XT4 Kraken system at University of Tennessee Knoxville, while a similar run on the 28K processor Jaguar system at ORNL, which has more memory per processor, sustained 35.7 teraflops (a higher flops rate) with a 1.94 shortest period. For the final run we obtained access to the ORNL Petaflop System, a new very large Cray XT5, and achieved 1.72 shortest period and 161 teraflops using 149,784 processor cores.